Modeling Time Varying Systems Using Hidden Control Neural Architecture

Multi-layered neural networks have recently been proposed for nonlinear prediction and system modeling. Although proven successful for modeling time invariant nonlinear systems, the inability of neural networks to characterize temporal variability has so far been an obstacle in applying them to complicated non stationary signals, such as speech. In this paper we present a network architecture, called "Hidden Control Neural Network" (HCNN), for modeling signals generated by nonlinear dynamical systems with restricted time variability. The approach taken here is to allow the mapping that is implemented by a multi layered neural network to change with time as a function of an additional control input signal. This network is trained using an algorithm that is based on "back-propagation" and segmentation algorithms for estimating the unknown control together with the network's parameters. The HCNN approach was applied to several tasks including modeling of time-varying nonlinear systems and speaker-independent recognition of connected digits, yielding a word accuracy of 99.1 %.